Wi score #561
Wi score #561
Conversation
|
Hello @dajmcdon 👋 If you are able to provide me with a working example of inputs and output, then I can set up the basic scaffolding for a metric |
|
@EmilHvitfeldt This is what should happen. My last commit removed the portions of this PR that are now in The below are the Examples for that function (but using a different backing), so this won't run as written. [I used Is this what you had in mind? library(yardstick)
library(hardhat)
quantile_levels <- c(.2, .4, .6, .8)
predq1 <- 1:4 #
predq2 <- 8:11
dstn <- quantile_pred(rbind(predq1, predq2), quantile_levels)
actual <- c(3.3, 7.1)
weighted_interval_score(dstn, actual)
#> [1] 0.65 1.90
weighted_interval_score(dstn, actual, c(.25, .5, .75))
#> [1] 0.6833333 1.9833333
# Missing value behaviours
dstn <- quantile_pred(matrix(c(1, 2, NA, 4), nrow = 1), 1:4 / 5)
weighted_interval_score(dstn, 2.5)
#> [1] 0.5
weighted_interval_score(dstn, 2.5, 1:9 / 10)
#> [1] 0.455656
weighted_interval_score(dstn, 2.5, 1:9 / 10, na_handling = "drop")
#> [1] 0.462613
weighted_interval_score(dstn, 2.5, na_handling = "propagate")
#> [1] NA
weighted_interval_score(
quantile_pred(matrix(1:4, nrow = 1), 1:4 / 5),
actual = 2.5,
quantile_levels = 1:9 / 10,
na_handling = "fail"
)
#> [1] NACreated on 2025-12-03 with reprex v2.1.1 |
|
yes that is perfect! I'll set up the scaffolding so it runs correctly, then I'll ping you for documentation and correctness testing |
|
I just realized some of the functionality above does not match the arguments. Let me fiddle with things a bit. |
|
no problem! let me know when you are ready for me to take the reigns |
|
OK. Should be good to go now. Here's a revised version of the examples (using library(yardstick)
library(hardhat)
quantile_levels <- c(.2, .4, .6, .8)
pred1 <- 1:4
pred2 <- 8:11
preds <- quantile_pred(rbind(pred1, pred2), quantile_levels)
truth <- c(3.3, 7.1)
weighted_interval_score_vec(truth, preds)
#> [1] 1.275
weighted_interval_score_vec(truth, preds, quantile_levels = c(.25, .5, .75))
#> [1] 1.333333
# Missing value behaviours
preds_na <- quantile_pred(rbind(pred1, c(1, 2, NA, 4)), 1:4 / 5)
truth <- c(2.5, 2.5)
weighted_interval_score_vec(truth, preds_na)
#> [1] 0.5
weighted_interval_score_vec(truth, preds_na, quantile_levels = 1:9 / 10)
#> [1] 0.455656
weighted_interval_score_vec( # should error
truth,
preds_na,
quantile_levels = 1:9 / 10,
quantile_estimate_nas = "drop"
)
#> Error in `weighted_interval_score_vec()`:
#> ! When `quantile_levels` is not a subset of those available in
#> `estimate`, `quantile_estimate_nas` may not be `'drop'`.
weighted_interval_score_vec(
truth,
preds_na,
quantile_levels = c(2, 3) / 5,
quantile_estimate_nas = "drop"
)
#> [1] 0.4
weighted_interval_score_vec(
truth, preds_na, na_rm = TRUE, quantile_estimate_nas = "propagate"
)
#> [1] 0.5
weighted_interval_score_vec(
truth, preds_na, na_rm = FALSE, quantile_estimate_nas = "propagate"
)
#> [1] NACreated on 2025-12-03 with reprex v2.1.1 |
2 participants
No description provided.